IGR Report for EPSRC Grant GR/R64742/01 A Rigorous Investigation into Estimation of Distribution Algorithms

Estimation of Distribution Algorithms (EDAs) have been recognized as a major paradigm in evolutionary computation. There is no traditional crossover or mutation in EDAs. Instead, they explicitly extract global statistical information from the selected solutions (often called parents) and build a posterior probability distribution model of promising solutions, based on the extracted information. New solutions are sampled from the model thus built and fully or in part replace the old population. Since the dependence relationships in the distribution of the promising solutions are highly relevant to the variable interactions in the problem, EDAs are promising methods for capturing the structure of variable interactions, identifying and manipulating crucial building blocks, and hence efficiently solving hard optimization and search problems with interactions among the variables. Many EDA-like algorithms have been developed for various optimization and search problems in recent years. Instances of EDAs include Population-Based Incremental Learning (PBIL), Univariate Marginal Distribution Algorithm (UMDA), Mutual Information Maximization for Input Clustering (MIMIC), Combining Optimizers with Mutual Information Trees (COMIT), Factorized Distribution Algorithm (FDA), Bayesian Optimization Algorithm(BOA), Bayesian Evolutionary Algorithm (BEA), and Global Search Based on Reinforcement Learning Agents (GSBRL), to name a few [1]. Relatively little effort has been devoted to studying the working mechanisms of EDAs. Mühlenbein [3], González et al. [2] and Höhfeld & Rudolph [4] have studied the behaviours of UMDA and PBIL (the simplest versions of the EDA, which ignore all the variable interactions). Their results show that these algorithms can locate the optimum of a linear function but cannot solve problems with nonlinear variable interactions. In [5], Mühlenbein and Mahning discussed the convergence of FDA (Factorized Distribution Algorithm) for separable additively decomposable functions (ADFs). Since there are no overlaps in their objective functions, their FDA is equivalent to UMDA. Therefore, their work does not deal with the ability of FDA to solve problems with variable interactions. The theoretical study of the ability of EDAs for dealing with variable interactions is urgently needed in order to obtain a deeper understanding of EDAs. Since it is impractical to calculate the actual posterior distribution of the promising solutions, most of the existing EDA-like algorithms model the distribution functions by probabilistic graph models or Bayesian networks. These algorithms can only take into account some selected dependence relationships that satisfy the triangulation constraints. This inherent shortcoming severely limits the ability of the algorithms in solving hard problems with other interaction structures. Besides, EDAs mainly explore the search space by random sampling from probability models. Therefore, EDAs in themselves are often very computationally expensive. Most, if not all, researchers currently use conventional local search techniques in EDAs for overcoming these shortcomings. Any other systematic methods for improving the performance of EDAs should benefit the applications of EDAs. The first part of this project is on the theory of EDAs. We have theoretically studied the behaviours of two typical EDAs: UMDA and FDA, under widely-used selection schemes. These two algorithms can also be regarded as instances of ant colony optimization methods. We have shown that it is necessary and sufficient, in terms of convergence, to consider some selected crucial dependence relationships in EDAs for optimization of additively decomposable functions. These theoretical results provide a real insight into the working mechanisms of EDAs. In the second part of this project, we have developed an efficient implementation of an EDA for global